We present a novel approach that unifies neural operators, physics-informed machine learning, and traditional numerical methods for solving PDEs within a single, versatile framework. Our method, Finite Operator Learning (FOL), enables parametric, data-free solutions to PDEs while providing accurate sensitivities for gradient-based optimization—without the high costs typically associated with adjoint methods. By leveraging a feed-forward neural network, FOL directly maps the parametric input space to the solution space, ensuring compliance with physical laws through customized loss functions. We employ the Finite Element Method (FEM) to approximate fields and their spatial derivatives and use Sobolev training to optimize a multi-objective loss function. Our framework is applied to the heat equation and mechanical equilibrium problems, demonstrating its effectiveness in learning material properties in heterogeneous systems, varying source terms, and different boundary conditions. Complex shapes and geometries of structures can also be easily handled thanks to the integration of FEM meshes into the proposed deep learning framework. Additionally, we explore Fourier-based parameterization to efficiently generate complex patterns with a limited number of variables. This method serves both as an advanced tool for optimization and parametric learning, as well as a matrix-free PDE solver, showing competitive performance with traditional solvers on nonlinear problems.